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(54) Speech translator for the deaf 



(57) A speech translator has a phoneme recogniser for detecting phonemes and a display for displaying 
them. An interface communicates between the recogniser and a telephone line. The phonemes recogniser 
may be provided in the user's home. Alternatively a plurality of phoneme recognisers may be provided in a 
central telephone office or in the office of an independent service provider. The user may be provided with 
a keypad, for reply, coupled to a pre-recorded announcement module and a text-to-speech module. 
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SPECIFICATION 

Speech translator for the deaf 

5 The invention relates to a communication de- 
vice for the deaf. 

Telecommunications Devices for the Deaf 
(TDDs) are known which provide the capability 
for someone at the other end of a telephone 

10 connection to type information on a keyboard. 
This information is encoded and transmitted 
over the telephone connection by one of sev- 
eral means and arrives at the TDD belonging 
to the deaf person. There it is decoded and 

1 5 displayed or printed for the deaf person to 
read. The deaf person can respond in a similar 
manner. 

For a deaf person to carry on a conversa- 
tion with a party over the telephone network, 

20 that other party as well as the deaf person 
must have a TDD. TDDs are relatively expen- 
sive, and are not in common use. Thus, a 
deaf person cannot communicate with most of 
the population via the telephone. 

25 The invention, incorporated into a device en- 
titled the Speech Translator for the Deaf 
(STD), overcomes these limitations by translat- 
ing the speech sounds coming in over the 
telephone connection to symbolic text which 

30 is displayed to the deaf user. The deaf user 
can then recognise this symbolic text as 
words. This symbolic text is a phonetic repre- 
sentation of the incoming sounds, thereby 
looking like the spoken words. For example: 

35 the spoken word "Boston"; might appear to 
the deaf user as "baahstun". 

In order that the invention and its various 
other preferred features may be understood 
more easily, some embodiments thereof will 

40 now be described, by way of example only, 
with reference to the drawings in which: 

Figures 1 and 2 are basic schematic draw- 
ings which show use of the invention in a 
preferred embodiment, 

45 Figure 3 is a block schematic diagram 
showing an implementation of the invention, 

Figures 4 and 5 show schematically portable 
alternative embodiments constructed in accor- 
dance with the invention, and 

50 Figures 6, 7 and 8 are block schematic dia- 
grams which show alternative embodiments 
formed as a unit located remotely from the 
user. 

Figures 1 and 2 depict the STD in a deaf 
55 user's residence connected to the telephone 
network with a hearing party at the other end 
of the connection. A deaf user of the STD 
who has never heard speech may require 
some training to associate the symbolic text 
60 with words, but this would still be consider- 
ably easier than learning lip reading. Deaf us- 
ers who at one time could hear should require 
little or no training in the use of the STD. 
The deaf user has the capability, either user 
65 is also mute, the STD would optionally con- 



vert text which has been typed into the key- 
board to mechanical speech which is then out- 
put to the hearing party. This would be done 
using an optional Text-to-Speech module. 

70 Also, pre-recorded, canned phrases can be 
provided which are activated by simple com- 
mands from the keyboard. 

The particular implementation described here 
interfaces directly to the telephone network, 

75 and is described with reference to Figure 3. 
The Telephony Interface 1 interfaces to the 
telephone line. It provides notification of in- 
coming ringing and an idle/seized status indi- 
cation to an Output Controller 6. It also re- 

80 sponds to commands from a Keypad Interface 
9 to answer or seize the telephone line, by 
providing loop resistance for current to flow 
from the serving telephone office, and to gen- 
erate address or dial pulses on the telephone 

85 line, by interrupting the loop current. 

The Hybrid 2 is a 2-wire to 4-wire circuit to 
separate the incoming speech or tones from 
the outgoing speech or tones. 
The Audio Monitor 3 is a speaker which 

90 can be switched in or out so the conversation 
from the far end can be monitored by some- 
one who can hear at the location of the deaf 
user. 

The Tone Detector 4 detects special tones 
95 coming from the telephone network such as 
ringback tone, busy tone, reorder (fast busy) 
tone, and dial tone. As an option to provide a 
lower level of communication in case a pho- 
neme recogniser 5 fails, or an accent or 
100 speech impediment makes recognition difficult, 
DTMF (dual tone multi-frequency) tone detec- 
tors may also be provided. The outputs of the 
Tone Detector are sent to the Output Control- 
ler 6. 

105 The Phoneme Recogniser 5 is programmed 
to recognise small parts of speech in the in- 
coming words. From 40 to 150 of these 
small distinct parts or phonemes would, for 
example, be used. The output of the Phoneme 

1 10 Recogniser is passed to the Output Controller 
6. 

The Output Controller 6 controls a display 
7. It takes the outputs from the Tone Detec- 
tor 4 and the Telephony Interface 1 and dis- 

115 plays, either on an alpha-numeric display or 
labelled indicator lamps, the status of the tele- 
phone connection along with any applicable 
messages having to do with operation of the 
device. It also takes the output of the Pho- 

120 neme Recogniser 5 and converts this output 
to symbolic text which is displayed on the 
alpha-numeric display 7 for the deaf user to 
read and interpret. 
The Display 7 consists of an alpha-numeric 

125 display on which the translated symbolic text 
and any applicable operation control messages 
are presented. Also associated with the Dis- 
play there are individual indicator lamps asso- 
ciated with control of the device. 

130 A Keypad 8 is the means by which the deaf 
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user both controls the device itself and, in 
case the deaf user is also mute, communi- 
cates with the hearing party at the far end of 
the telephone connection. It consists of an al- 
5 pha-numeric keypad with various special func- 
tion keys and switches. The Keypad outputs 
are directed to the Keypad Interface 9. 

The Keypad Interface 9 interprets com- 
mands from the Keyboard 8 and directs them 

10 to other blocks of this system such as a 
DTMF (or Touch-Tone) Generator 10 which 
can generate DTMF address signals for routing 
to the telephone network; a Pre-Recorded An- 
nouncements module 1 1 which can generate a 

1 5 prerecorded announcement for routing to the 
telephone network; a Text-to-Speech Module 
12 which can convert the ASCII text from the 
keyboard to speech and route it onto the tele- 
phone network; and the Telephony Interface 1 

20 which would seize or answer the telephone 
line or send out addressing dial pulses. 

The DTMF Generator 10 generates DTMF 
address signals which are routed to the tele- 
phone network in response to commands 

25 from the Keypad 8. 

The Pre-Recorded Announcement Module 1 1 
generates pre-recorded announcements which 
are routed to the telephone network in re- 
sponse to commands from the Keypad 8. This 

30 module could be optional as some deaf users 
have no speaking problem and would not 
need this capability. This would allow standard 
phrases to be played in response to short 
commands from the Keypad. Examples of 

35 such phrases are as follows: "Hello, please 
speak slowly and distinctly as you are speak- 
ing into a device to translate for a deaf per- 
son". 

"Please repeat that again". 
40 "Yes", "No", "Okay", etc. 

The Text-to-Speech Module 12 translates 
ASCII text from the Keypad into speech and 
routes it onto the telephone network. This 
module could be optional as some deaf users 
45 have no trouble speaking and would not need 
this capability. 

A Microphone and Amplifier 13 are pro- 
vided to allow someone to speak directly onto 
the telephone line. This also, in conjunction 
50 with a Test switch or relay 14, allows for test 
and verification that the system is functioning 
properly. 

The Test switch or relay 14 provides for 
the transmit side of the system to be con- 

55 nected directly to the receive side for pur- 
poses of test and verification. 

The invention may also be employed for 
other applications. By coupling the Telephone 
Interface 1 with a microphone connected to 

60 the receive circuitry and a speaker connected 
to the transmit circuitry, this device could be 
used to carry on a face-to-face conversation 
between a deaf user and another person who 
speaks into the microphone. The deaf user 

65 observes and interprets the symbolic text on 



the display in the same manner as with the 
other applications. This could be a portable 
unit. Figure 4 illustrates this implementation. 
By replacing the Telephony Interface 1 with 

70 an acoustic coupler, this device could become 
a portable unit which could be carried about 
and used at pay telephones, hotels, etc. by 
placing the telephone handset in the acoustic 
coupler. The unit does not then have to be 
75 connected directly to the telephone line. Figure 
5 illustrates this implementation. 

The STD, or several of them, could be 
placed in a central location, such as a tele- 
phone office, and its use leased on a daily, 

80 weekly, or monthly basis. To accomplish this, 
the Display and Keypad resides in the deaf 
user's home and can be connected when 
needed to the main unit over a dial-up data 
communication with data modems. Several of 
85 these units can be supplied in the central loca- 
tion with switched access to telephone lines 
both on the hearing party side and on the 
deaf user side which would have the remote 
data capability via data modems. This may 
90 require some intelligence in the switches be- 
tween the telephone lines and both sides of 
the STDs to accomplish the connection. 

Figure 6 shows the centralised service pro- 
vided in the telephone office with the data 
95 connection between the STD in the central 
office and the keyboard/display in the deaf 
user's residence accomplished using standard 
data modems. This may require the deaf user 
to have two telephone lines to his home. 

100 Figure 7 shows the service still being pro- 
vided in the telephone office, but here the 
connection to the deaf user's home is over a 
single voice/data line, for example, an LADT 
or an ISDN type of line. 

105 Figure 8 shows the service being provided 
by an independent service provider who has 
several voice/data lines leased into his facility 
from the telephone office. This may require 
the served deaf user to have one normal tele- 

1 10 phone line and one voice/data type line to his 
home. This same service could be provided 
by replacing each voice/data line with two 
normal telephone lines and using standard mo- 
dems to transmit data. 

1 15 The STD presents to the user not only the 
symbols representing the voiced sounds, but 
also indications as to the status of the tele- 
phone line (incoming ringing, busy tone, dial 
tone, ringback tone, reorder tone, DTMF di- 

120 gits, etc.). The STD also provides the capabil- 
ity of generating speech from typed text or to 
generate a pre-designated speech message so 
that a deaf user who is also mute can com- 
municate back to the party at the other end of 

125 the connection. The STD allows several 

methods of centralising this translation func- 
tion at some common facility which could pro- 
vide service to a number of users. 
The phoneme recogniser of the exemplary 

130 system detects the phonemes, from a defined 
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repertoire, that are present in the speech 
which is presented to its input and to present 
at its output some code, in real time, to the 
external circuitry indicating the presence of 
5 those phonemes. 

A phoneme is a distinctive speech sound 
which, when coupled with other phonemes, 
form words. Words in the English language 
can be represented by as few as around 50 

10 phonemes. For better clarity more than 100 
can be used. A typical two-syllable word 
might consist of 6 to 12 phonemes depending 
upon the repertoire of defined phonemes. 
A phoneme recogniser which may be em- 

15 ployed in the STD is described, for example, 
in U.S. Patent 4,181,813 to John Marley, 
which employs delta modulation techniques to 
distinguish different phonemes. A system 
which embodies the invention does not require 

20 entire words to be recognised by the equip- 
ment — this is a function of the human brain 
putting the phonemes together. 

CLAIMS 

25 1. A communication device, comprising a 
phoneme recogniser, for detecting phonemes 
present in speech, display means, coupled to 
the recogniser, for displaying the detected 
phonemes, means, coupled to the recogniser, 

30 to interface the recogniser to a telephone line, 
and means, coupled to the display means and 
to the phoneme recogniser for controlling the 
display means. 
2. A communication device as claimed in 

35 claim 1 , comprising a 2-wire to 4-wire hybrid 
circuit, coupled to the input of the phoneme 
recogniser, for separating incoming signals of 
the device from outgoing signals of the de- 
vice. 

40 3. A communication device as claimed in 
claim 2, comprising a test switch, coupled to 
the hybrid circuit, for connecting the outgoing 
signals to the incoming signal output of the 
hybrid circuit for device test and verification. 

45 4. A communication device as claimed in 
claim 1, 2 or 3, comprising means, coupled to 
the display control means, for detecting tones 
coupled thereto from a telephone line. 

5. A communication device as claimed in 
50 claim 4, wherein indications of the tones are 

displayed by the display means to provide in- 
dications as to the status of the telephone 
line. 

6. A communication device as claimed in 
55 any one of the preceding claims, comprising 

tactile control means, coupled to the tele- 
phone line interface, for providing device con- 
trol to a user of the device. 

7. A communication device as claimed in 
60 claim 6, wherein the tactile control means 

comprises a keypad, for providing the user 
with tactile control input means, and an inter- 
face, coupled between the keypad and the 
telephone line interface, to provide an inter- 
65 face therebetween. 



8. A communication device as claimed in 
any one of the preceding claims, comprising 
means, coupled to the telephone line interface 
means, for providing outgoing address signals 

70 to the telephone line. 

9. A communication device as claimed in 
any one of the preceding claims, comprising 
means, coupled to the telephone line interface 
means, for providing pre-recorded outgoing 

75 announcements to the telephone line. 

10. A communication device as claimed in 
any one of the preceding claims, comprising 
an audio monitor, switchably coupled to the 
input of the phoneme recogniser, for providing 

80 audible monitoring of speech. 

1 1 . A communication device as claimed in 
any one of the preceding claims, comprising 
means, coupled to the telephone line interface 
means, for translating user-coded text into 

85 user-supplied speech and providing the user 
supplied speech to the telephone line. 

12. A communication device as claimed in 
any one of the preceding claims, comprising a 
microphone, coupled to the telephone line in- 

90 terface, for providing outgoing speech to the 
telephone line. 

13. A communication device as claimed in 
any one of the preceding claims, located re- 
motely from a user of the device. 

95 14. A communication system, comprising at 
least one phoneme recogniser, for detecting 
phonemes present in speech; and means, 
coupled to said phoneme recogniser, for pro- 
viding indications of said detected phonemes 
100 via communication links to displays located re- 
motely from said phoneme recogniser. 

15. A communication device or system sub- 
stantially as described herein with reference to 
the drawings. 

105 

CLAIMS 

Amendments to the claims have been filed, 
and have the following effect: 
New or textually amended claims have been 
1 10 filed as follows: 

16. A communication device as claimed in 
claim 4 wherein the means for detecting tones 
are coupled to the means to interface, and the 
display means are controlled by the means for 

1 1 5 controlling as a function of outputs produced 
by the means for detecting tones and the 
phoneme recognizer. 

17. A communication device in accordance 
with claim 2 wherein the 2-wire to 4-wire hy- 

120 brid circuit has incoming and outgoing signal 
portions. 

18. A communication device in accordance 
with claim 17 further comprising a test switch 
for connecting the outgoing signal portions to 

125 the incoming signal portions of the hybrid cir- 
cuit for device test and verification. 
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